Skip to content

fix(typegen): accept deserialized json/jsonb values in generated types for Python - #1129

Closed
h-zahar wants to merge 1 commit into
supabase:masterfrom
h-zahar:fix/typegen-python-json-mismatch
Closed

fix(typegen): accept deserialized json/jsonb values in generated types for Python#1129
h-zahar wants to merge 1 commit into
supabase:masterfrom
h-zahar:fix/typegen-python-json-mismatch

Conversation

@h-zahar

@h-zahar h-zahar commented Aug 31, 2026

Copy link
Copy Markdown

The issue was reported in supabase/supabase-py#1597.

Currently, the Python generator maps json and jsonb columns to pydantic's Json[Any], which expects a str containing JSON and deserializes it. PostgREST returns those columns already deserialized, so every generated model with a JSON column raises on model_validate: JSON input should be string, bytes or bytearray.

With this PR, both column types map to pydantic.JsonValue, which denotes an already-deserialized JSON value.

The issue is resolved using pydantic's native type without defining a custom type.

@h-zahar
h-zahar requested review from a team, avallete and soedirgo as code owners August 31, 2026 03:55
@h-zahar h-zahar changed the title fix(typegen): accept deserialized json/jsonb values in generated types for python fix(typegen): accept deserialized json/jsonb values in generated types for Python Aug 31, 2026
@spydon

spydon commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Thank you for the contribution! postgres-meta's type generation is moving to the shared @supabase/postgrest-typegen package in supabase/sdk (see #1084), so open template fixes are being re-landed there. Your fix (mapping json/jsonb to pydantic's JsonValue so deserialized PostgREST payloads validate) has been ported in supabase/sdk#124 with credit to this PR.

@spydon spydon closed this Aug 31, 2026
spydon added a commit to supabase/sdk that referenced this pull request Sep 1, 2026
…thon

pydantic's Json[Any] validates a JSON string and deserializes it, but
PostgREST returns json and jsonb columns already deserialized, so every
generated model with a JSON column failed model_validate. JsonValue is
pydantic's type for an already parsed JSON value.

Ported from supabase/postgres-meta#1129
spydon added a commit to supabase/sdk that referenced this pull request Sep 1, 2026
…columns, composite nullability (#124)

## Summary

Ports the worthwhile Python generator fixes from postgres-meta's open
template PRs into this package (the templates are being deleted in favor
of this package in supabase/postgres-meta#1084, so open fixes there are
triaged and re-landed here). Four fixes, one commit each:

1. **Identifier escaping** (from supabase/postgres-meta#1082): enum
`Literal` labels and `Field(alias=...)` values were interpolated
unescaped, so a quote, backslash, or newline in a database name broke
the generated module. A shared `escapePythonString` helper (JSON
escaping, a strict subset of Python's) now covers all three
interpolation sites.
2. **Python 3.9/3.10 support** (from supabase/postgres-meta#1094):
`NotRequired` (3.11+) and `TypeAlias` (3.10+) now import from
`typing_extensions`, which is always installed as a required dependency
of pydantic.
3. **Deserialized json/jsonb** (from supabase/postgres-meta#1129):
`json`/`jsonb` map to pydantic's `JsonValue` instead of `Json[Any]`.
PostgREST returns these columns already deserialized, while `Json[Any]`
validates a JSON *string* and parses it, so every generated model with a
JSON column failed `model_validate` (supabase/supabase-py#1597).
4. **Composite type nullability** (the Python side of
supabase/postgres-meta#1063, reimplemented): composite type attributes
cannot carry NOT NULL constraints in Postgres, so their fields now emit
`Optional[...]`. The origin PR's Python hunks were dead code (an unused
`PythonDomain` class and a type-map entry for a name Postgres never
emits), so the actual fix was implemented instead of ported.

## Triage of origin PRs

| postgres-meta PR | Verdict | Reasoning |
|---|---|---|
| #1082 | Ported | Real invalid-syntax bug, correct approach. |
| #1094 | Ported | Import failure on Python 3.9/3.10, independently
verified by community comments on the PR. |
| #1129 | Ported | Every JSON column failed validation at runtime;
`JsonValue` is pydantic's native type for a parsed JSON value. |
| #1063 (python part) | Reimplemented | Real bug, but the PR's Python
changes did not actually fix it (dead code); the underlying fix is one
line in `typeToClass`. |
| #1072 (`frozen=True`) | Skipped | Author-labeled feature and an
opinionated behavior change that breaks consumers who mutate row models;
belongs behind a generator option if wanted. |
| #808 | Skipped | 2023 draft fully superseded by the
maintainer-authored template this package ports. |

## Validation

- Unit tests per fix (pathological enum labels and aliases, import block
assertions, json/jsonb mapping, composite `Optional` fields).
- Parity golden regenerated (39 lines): the `typing_extensions` import
split, 16 `Json[Any]` to `JsonValue` occurrences, and two composite
attributes gaining `Optional[...]`; reviewed line by line and the golden
gate was verified to actually trip on corruption.
- The regenerated golden imports cleanly under pydantic, passes `mypy`,
and runtime checks confirm deserialized JSON and `None` composite fields
now validate.
- `check-types`, `format-and-lint`, `knip`, `build`, `test` (97 pass,
includes Docker-backed introspection and parity) all green.
- Note: the nightly parity job against real postgres-meta will show this
intentional drift until postgres-meta consumes a release containing it
(supabase/postgres-meta#1084 replaces the templates with this package,
closing the gap).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants